Topic Significance Ranking of LDA Generative Models

Topic models, like Latent Dirichlet Allocation (LDA), have been recently used to automatically generate text corpora topics, and to subdivide the corpus words among those topics. However, not all the estimated topics are of equal importance or correspond to genuine themes of the domain. Some of the topics can be a collection of irrelevant or background words, or represent insignificant themes. Current approaches to topic modeling perform manual examination of their output to find meaningful and important topics. This paper presents the first automated unsupervised analysis of LDA models to identify and distinguish junk topics from legitimate ones, and to rank the topic significance. The basic idea consists of measuring the distance between a topic distribution and a ”junk distribution”. In particular, three definitions of ”junk distribution” are introduced, and a variety of metrics are used to compute the distances, from which an expressive figure of topic significance is implemented using a 4-phase Weighted Combination approach. Our experiments on synthetic and benchmark datasets show the effectiveness of the proposed approach in expressively ranking the significance of topics.